Skip to content

Conversation

@sayakpaul
Copy link
Member

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 5, 2023

The documentation is not available anymore as the PR was closed or merged.

@ziniuwan
Copy link

ziniuwan commented Oct 5, 2023

Thanks! It solves the bug of SD pipeline, but seems like SDXL pipeline is still not working.
Colab notebook

@patrickvonplaten
Copy link
Contributor

Nice, that's an important fix! Let's merge 🚀

BTW this bug might be solved natively on Torch 2.1 now.

@patrickvonplaten
Copy link
Contributor

Also cc @younesbelkada @pacman100 here. Having torch.compile work with PEFT + diffusers would be important

@sayakpaul
Copy link
Member Author

@ziniuwan weird that it happens in specific cases. For example, if you call the pipeline (where the UNet is already compiled) and then load the LoRA params and call the pipeline, it works:

https://colab.research.google.com/gist/sayakpaul/58d9640998be5e1852ca630a7b006a8c/reproduce_lora_sdxl_bug.ipynb

@patrickvonplaten have you faced #5298 (comment) before?

@sayakpaul
Copy link
Member Author

@ziniuwan we're merging this PR. For SDXL, we suggest using fuse_lora() to achieve speedup (this is also a general suggestion in case you want to speed up the LoRA computations).

fuse_lora() docs:

https://huggingface.co/docs/diffusers/main/en/training/lora#fusing-lora-parameters

@sayakpaul sayakpaul merged commit 7eaae83 into main Oct 6, 2023
@sayakpaul sayakpaul deleted the fix/torch-compile-lora branch October 6, 2023 15:15
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_lora_weight does not work with torch.compile

5 participants